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Abstract 

We will study a population of individuals playing the infinitely 
repeated Prisoner’s Dilemma under replicator dynamics. The popula¬ 
tion consists of three kinds of individuals using the following reactive 
strategies: ALLD (individuals which always defect), ATFT (almost 
tit-for-tat; individuals which almost always repeat the opponent’s last 
move) and G (generous individuals, which always cooperate when the 
opponent cooperated in the last move and have a positive probability 
q of cooperating when they are defected). Our aim is studying in a 
mathematically rigorous fashion the dynamics of a simplified version 
for the computer experiment in [12] involving 100 reactive strategies. 
We will see that as the generosity degree of the G individuals varies, 
equilibria (rest points) of the dynamics appear or disappear, and the 
dynamics changes accordingly. Not only we will prove that the results 
of the experiment are true in our simplified version, but we will have 
complete control on the existence or non-existence of the equilbria 
for the dynamics for all possible values of the parameters, given that 
ATFT individuals are close enough to TFT. For most values of the 
parameters the dynamics will be completely determined. 
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1 Introduction 

We know that cooperation either between individuals, or between parts, ex¬ 
ists not only in human societies, but, more in general, in all biological sys¬ 
tems. Cooperating individuals usually have to pay a cost for the benefit of 
other individuals. It is then an interesting question to understand how co¬ 
operation can evolve in the light of Darwinian natural selection. Sigmund, 
Nowak and collaborators have studied in several contributions the evolution 
of cooperation, see e.g. [8] and [9] for some basic information and more 
references. 

The essence of the problem can be grasped by the famous Prisoner’s 
Dilemma (PD) stated in different forms by [13] and [H]. In mathematical 
terms, the PD is characterized by only two strategies, a pay-off matrix 
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and the pay-off rankings T > R > P > S. Strategy 1 is C (cooperate) 
and strategy 2 is D (defect); matrix element aij is the pay-off received by 
an individual playing strategy i confronted by an individual playing strategy 
j, i,j G {1,2}. With the above ranking, given the opponent’s strategy, 
defection always has a larger pay-off than cooperation. Using the jargon of 
game theory |8|, in the PD strategy D is a strict Nash equilibrium, whereas 
C is not a Nash equilibrium. Rational individuals must choose D in the PD. 

A window for cooperation may be opened when individuals can perceive 
that pay-off R for mutual cooperation is better than P for mutual defection. 
This cannot happen for the simple PD above, because individuals will interact 
only once. If individuals are given the opportunity of interacting many times 
before they receive their pay-offs, then a reciprocity mechanism can favor 
cooperation. But in this case C and D are not the only possible strategies 
and we have the problem of selecting among a huge number of strategies 
combining C and D. 

In the 1970’s [1] studied strategies for the repeated PD. He organized two 
tournaments of the repeated PD and in both realizations of the tournament 
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the winner strategy was the simplest among all submitted strategies: tit-for- 
tat (TFT). TFT is the strategy which repeats the previous movement of its 
opponent. 

In Evolutionary Game Theory, see |5] and [1], pay-offs are viewed as 
biological htness and strategies in a population having larger pay-offs have 
the tendency to increase their frequencies. Suppose we have a population of 
individuals playing n strategies si,..., and let Xi{t) be the fraction of the 
population occupied at time t by individuals playing strategy s*. Let also x 
denote the vector (xi,..., Xn)- We dehne the htness of strategy Sj as 

fi{x) = {Ax)i , (2) 

the i-th element of a matrix product, where now A is an n x n pay-off matrix. 
The mean htness of the population is then 

n 

(j) = fi{x)xi . (3) 

i=l 

Finally, population dynamics is naturally given by a system of ordinary dif¬ 
ferential equations (ODEs) of the form 

Xi = {fi{x) - (j))xi , (4) 

i = 1,2,... , 77 ,, which were introduced in [15] and are called replicator dy¬ 
namics equations. It can be shown |3] that the simplex 

n 

S'n = {x e M" ; Xj > 0, ^ Xi = 1} 

i=l 

is invariant under the replicator dynamics. 

From now on we will consider the inhnitely repeated Prisoner’s Dilemma 
(IRPD) i.e. the limiting case tc = 1 in the repeated PD, and as admissible 
only reactive strategies dehned below: 

Definition 1 An individual adopting reactive strategy r{p, q) will choose be¬ 
tween C or D at some round based only on his opponent’s choice at the round 
before, according to the following stochastic rule: choose C with probability p 
if the opponent chose C the round before; choose C with probability q if the 
opponent chose D the round before. 
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We may think of p G [0,1] as a loyalty parameter and of g G [0,1] as a 
forgiveness parameter. Some simple strategies are recognized as reactive: 
ALLD, individuals which always defect, is denoted as r(0, 0) and TFT is 
r(l,0). 

Reactive strategies were dehned in HDl and further studied in and un¬ 
it can be shown m that if 

\{p - q){p'- q')\ < I (5) 

then the mean pay-off per round for each player may be dehned in terms of 
the equilibrium distribution of a Markov chain. This mean pay-off per round 
dehnes the deterministic IRPD pay-off E{s,s') for a player with reactive 
strategy s = r{p, q) against a player with reactive strategy s' = r(p', q')\ 

E{s, s') = Gicc' + (5 - P)c + (T - P)c' + P , (6) 

where 

_ (p - q)q' + q , _ {p' - q')q + q' 

O / \ / \ cLiJ-CJ. O , , ■. \ ) 

1 - (p - q){p' -q') 1 - (p - q)w - q') 

are respectively the equilibrium probabilities that s cooperates with s' and 
vice-versa, and 

Gi = {R-T) + {P - S) (8) 

is a parameter which will have great importance in this work. 

Notice that, apart other unimportant cases, condition ([^ is not satished 
in the case where s = s' = r(l, 0), i.e. both players are TFT. This is due to 
the fact that the outcome of the game between two TFT individuals must 
depend not only on their loyalty and forgiveness, but also on their initial 
moves: they will remain forever in the CC state when both play C at the 
hrst move, or in state DD when both play D, or alternate between states CD 
and DC in the remaining cases. As a consequence, pay-off R obtained by 
a perfect TFT against another perfect TFT if both start cooperating may 
become as low as P if arbitrarily small amounts of “noise” are present. 

Inspiration for this work was provided by the seminal computer exper¬ 
iment performed in na. In that paper authors took 99 reactive strategies 
randomly chosen in the square [0,1] x [0,1]. To that sample, suggested by 
Axelrod’s results, they added by hand strategy r(0.99, 0.01), an almost TFT 
(ATFT) strategy. TFT was not selected due to the above mentioned impos¬ 
sibility of dehning its pay-off. All 100 strategies were considered as having 
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equal fractions at the initial time and then evolution given by replicator dy¬ 
namics Q was numerically evaluated. Results were illustrated at Fig. 1 in 
na and are described as follows: 

1. Initially, strategies far from r(0,0) have their frequencies strongly de¬ 
pleted and it seems that the strategy closest to ALLD will extinguish 
all the others. 


2. After some time the frequency of ATFT starts increasing, and it looks 
like it will win the game. 

3. After a lot more time the ATFT frequency decreases and a surprising 
strategy named generous TFT (GTFT) hnally drives all other strate¬ 
gies to extinction. In the experiment of [12], performed with the same 
parameter values as Axelrod’s tournaments, i.e. T = 5, R = 3, P = 1, 
S = 0, the winner was the strategy closest to r(l, |). 

GTFT seems to have been discovered in |6| and rediscovered exactly in 
[T2|. Molander defined GTFT as strategy ^(1,^) with q close to 


,2R-S-T R-P^ 
qcTFT = mm{—^. 


(9) 


GTFT is what we may call a genuine collaborative strategy, altruistic indeed, 
as stated in |6|. GTFT does more than TFT in reciprocating cooperation, it 
is also forgiving. 

Our objective in writing this paper is to give precise mathematical ar¬ 
guments supporting the results found in [U] and in [T2]. Molander’s paper 
considers a situation in which there is no dynamics at all, only pairwise 
comparison between pay-offs obtained using different strategies in the IRPD. 
Moreover strategies considered in his paper are not reactive, but mixed strate¬ 
gies [3|. One important difference instead between this paper and [T2| is that, 
unable at this time to prove results for numbers of strategies as large as 100, 
we simplify their model and consider the IRPD with only the three more 
prominent strategies in their experiment. Moreover, our results will be valid 
for any suitable choice of the many parameters of the problem, not a hxed 
choice, and any initial conditions for the population. 

More concretely, we will consider a population of individuals under the 
replicator dynamics Q playing the IRPD (pay-offs calculated by ([^ and 
0 ) with three reactive strategies: 
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• Strategy 1 is an arbitrary ATFT, i.e. strategy r(l — € 1 , 62 ), where ei 
and €2 are positive and small enough. Differently of ca. we need not 
consider ei = 62 - 

• Strategy 2 is ALLD, i.e. r(0,0). 

• Strategy 3, which will be called Generous (G), is r(l,g), with q > 
(ei + 62 )^'^^, i.e. perfectly loyal individuals and more forgiving than the 
considered ATFT. 

We will prove the existence of a maximum amount of forgiveness qcTFT and 
a region of initial conditions with positive area such that, as in na, only 
strategy 3 will survive after infinite time. But also we will see what happens 
for larger values of q and find out that in some cases we may still have some 
weaker form of cooperation evolution. 

We will now define what we mean by weaker forms of cooperation evo¬ 
lution, so that we may state our results. For the sake of the following 
definition, we will suppose that the initial condition to be used in the dy¬ 
namics is random, taken with uniform probability among all possible triples 
(a:i( 0 ), 0 : 2 ( 0 ), 0 : 3 ( 0 )) in the simplex S^. 

Definition 2 We will say that the population admits 

• full evolution of cooperation if there is a positive probability that the 
dynamics will lead to extinction of all individuals adopting strategies 
ALLD and ATFT. 

• partial evolution of cooperation if there is a positive probability that the 
dynamics will lead to extinction of all individuals adopting ALLD, but 
ATFT and G will remain. 

• weak evolution of cooperation if there is a positive probability that the 
dynamics will lead to some condition where the G individuals are not 
extinct, but will coexist with ATFT and ALLD. 

• no evolution of cooperation if the dynamics leads to extinction of the G 
individuals with probability 1. 

The methods used are exact calculations of the pay-off matrix and anal¬ 
ysis of its entries. Some of the results depend on asymptotic analysis in 
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Table 1: The bullets indicate which equilibria are biological at each interval 
in the case Gi < 0. The red equilibrium and vertices Ei, E 2 and E 3 are 
biological at all intervals. Possible evolution of cooperation types are F (full), 
P (partial), W (weak) or N (no). A type followed by a ? means a conjectured 
result. 



Green Blue 

Black Diagrams 

Type 

((^1 “1“ ^ 2 ) ^ 5 Qgreen) 

• 

38R 

F 

iSlgreen^ Qblue) 

• • 

34R 

P 

i^Qbluei Qblack^ 

• 

36 

P 

i^Qblack') 1 ] 

• 

• 12, 12R, 13 

W? 


parameters ei and 62 - For each q we will determine all equilibria of the repli¬ 
cator dynamics in S 3 , study the dynamics at the boundary of S 3 and classify 
the few compatible phase portraits in the interior of that region, using for 
this classification results in [16] and [2]. In most cases only a single phase 
portrait of the complete table in [2] is compatible with the existent equilibria 
and dynamics at the boundary. In such cases the dynamics will be com¬ 
pletely determined. In some other cases more than one phase portrait in |2| 
will be found compatible, but although dynamics is not yet fully determined, 
we have some conjectures about it. 

The results are summarized in Tables [T|[^ and [^separated by the possible 
signs of Gi dehned by ([^. In any case we will define thresholds Qred, Qgreen, 
quue and quack and one equilibrium for the replicator dynamics with any of 
the above colors. For all possible intervals we will state which equilibria are 
present in the biological region S 3 . The red equilibrium will be biological in 
all cases, as well as points Ei, E 2 and E 3 . The tables also show which are the 
compatible phase portrait diagrams according to their numbering in [2] , and 
the kind of resulting cooperation evolution, if known. A letter R following 
the number of a diagram means that the phase portrait is the one in the 
diagram, but with all orbits having the reverse orientation. 

The paper is organized as follows. In Section we introduce the pay-off 
matrix, dehne the biological region, the red, green and blue lines and all 
the equilibria, with some important notations. In Section we prove some 
simpler properties of the entries of the pay-off matrix. Section [^contains the 
most important results of this paper, where appearance or disappearance of 
the equilibria in the biological region are calculated according to the sign of 
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Table 2: The bullets indicate which equilibria are biological at each interval 
in the case Gi > 0. The red equilibrium and vertices Ei, E 2 and E^ are 
biological at all intervals. Possible evolution of cooperation types are F (full), 
P (partial), W (weak) or N (no). A type followed by a ? means a conjectured 
result._ 



Green Blue Black 

Diagrams 

Type 

((ei + € 2 )^^^, QUack) 

• 

38R 

F 

i^QUackf Qblue) 

• • 

9R 

F 

i^Qblue') Qgreen) 

• 

15R 

N 

i^Qgreent 1 ] 

• • 

12, 12R, 13 

N? 


Table 3: The bullets indicate which equilibria are biological at each interval 
in the case Gi = 0. The red equilibrium and vertices Ei, E 2 and E^ are 
biological at all intervals. Possible evolution of cooperation types are F (full), 
P (partial), W (weak) or N (no). A type followed by a ? means a conjectured 
result._ 



Green Blue Black Diagrams 

Type 

((ei + 62 )^^^^, Qblack) 

• 38R 

F 

Qblack 

45 

F 

i.Qblack') 1 ] 

• • 12, 12R, 13 

W? 



Gi and the interval to which the forgiveness q of the G individuals belongs. 
Section relates the biological equilibria with the phase portraits in [2] and 
the corresponding types of evolution of cooperation. The paper is closed by 
a conclusions section. 


2 Pay-off matrix and notations 

Let T, R, P and S be the entries of the pay-off matrix for the simple PD, as 
in ([^. By dehnition of the simple PD, these parameters obey inequalities 

T > R> P > S . (10) 


We will also assume the following inequalities as hypotheses for the results 
in this paper: 

P<^<R. ( 11 ) 

The upper bound for (S' -|- T)/2 is a natural condition to ensure that alter¬ 
nating between C and D is not as good as a steady C for a pair of players, 
and has already appeared in [6] and further works. The lower bound seems 
to be a novel condition necessary for some of the proofs. As a consequence 
of this novel assumption we will have in Proposition that any amount of 
forgiveness in a reactive strategy will result in a pay-off larger than P for 
that strategy against itself. 

The pay-off matrix for the IRPD among the three strategies ATFT, ALLD 
and G may be calculated in a lengthy but straightforward fashion using ([^ 
and ([^. If strategies are numbered as in Section the result is 


A 


(1 


F(f^) (i-e^jP + e^S au(g) \ 

€ 2 )P + € 2 T P {l-q)P + qT 

031 (g) {l-q)P + qS R J 


where 


Fip) 


+ (^s + T)p + R 

a+py 


( 12 ) 


(13) 
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[g+(l-g)(ei + e2)]2 
ei 2R-S 


T+(T-i?)(6i + e2) 


? + (l-?)(ei + e2) 
R{l-e2)-eiS 
1-61-62 


6 l — 62 


( 14 ) 


and 


®3i(<?) — Gi 


l — ei-e2 


[g+ (1 -g)(ei + e 2)]2 
ei 2R- 


S-T-[ 6 ^{T-P) + 62 iR-S)] 


q + {1 - + (^ 2 ) 

R{1 - 62 ) - 6 iT 


61 - 62 


15) 


61 - 62 


In (14) and (|T^, Gi is the combination of parameters in 


It is known [3] that the simplex Sn is invariant nnder replicator dynamics 
(|^ for any nnmber n of strategies. In our case, we will always consider 
0:3 = 1 — a:i — X 2 , and accordingly call biological region the projection of the 
simplex S 3 onto the (xi,X 2 ) plane, i.e. the closed triangle B with vertices 
El = (1,0), E 2 = (0,1) and E 3 = (0,0). The sides of B will be denoted by 
Li, L 2 and L 3 , where Lj is the side on which Xj = 0 . 

Let the htnesses fi be dehned by ([^ and for i ^ j denote 


n,: 


= {(xi, X 2 ) e ; fi{xi, X 2 ,1 - xi - X 2 ) = fj{xi, X 2 ,1 - xi - X 2 )} 


the straight lines in which two htnesses are equal. We will also denote Pijk 
the point at which the line Uij intercepts the x*, = 0 line. Notice that using 
their dehnition above, coordinates for the Pijk can be easily calculated in 
terms of the entries in the pay-off matrix ( 12 ). 


From general arguments, see [3] , the equilibria for the replicator dynamics 
with three strategies can be: 

• Points in which only one strategy is present, i.e. the vertices i?i, E 2 
and E 3 oi B. 

• Points in which one strategy is absent and the other two have the same 
htness, i.e. P 123 , ^*132 and P 231 , whenever they exist. 
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• One point in which all three strategies have the same htness. This is 
the intersection of the three lines 77-12, and 7723, whenever it exists, 
and will be denoted Q. Notice that if two among these lines cross at a 
point, then the third lines must also pass through this point. 

Notice that replicator dynamics in everywhere well-dehned in and the 
above mentioned points are equilibria whether they he in B or not. The fact 
that we are only interested in the dynamics in B motivates the following 
dehnition: 


Definition 3 We will say that equilibria P123, P132 and P231 are biological 
whenever they lie in B, but not coincide with any of the vertices. We will 
say that equilibrium Q is biological whenever it lies in the interior of B. 

In the rest of this paper we will study whether each of the above mentioned 
equilibria is biological or not. These equilibria will be studied by locating 
the intersections of the nij lines with the sides of B and with each other. We 
found it simpler for the sake of the upcoming notations to give arbitrary color 
codes to each of the straight lines Uij and equilibria. Henceforth line 7712 will 
be referred to as the red line and P123, intersection of the red line with = 0, 
red equilibrium. Similarly, 7113 and P132 will be called respectively green line 
and equilibrium and 7123 and P231 will be called blue line and equilibrium. 
The coexistence equilibrium Q will be called black equilibrium. 

We will always be interested in positive values for ei and 62, and most 
of the results will hold given that these parameters are small enough. We 
dehne then polar coordinates r and 9 in the (61,62) plane, so that 

6i=rcos6' and 62 = 7 " sin 6 ^ ( 16 ) 


Throughout this paper, r and 9 will always be used with this meaning. 

Many times we will use e to refer to vector (ci, 62). We dehne the phrase 
“property P holds if e is small enough” as meaning “there exists ro > 0 such 
that property P holds if 0 < r < ro”. 

The overwhelming majority of the intermediate and hnal results in this 
paper will hold if 6 is small enough. From now on, as with ( 10 ) and 0 , we 
will assume as a hypothesis for the rest of this paper that e is small enough. 
In the beginning we will be explicit in stating this hypothesis, because we 
want the reader to be aware of it, but with time we will be increasingly more 
relapse in reminding it. 
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In some instances we will also use the notation 0(r“) standard in asymp¬ 
totic analysis. For not letting any doubt about it, if / is some function 
depending on e, we will write / = 0(r") if there exist rg > 0 and a constant 
K independent of r such that |//r“| < K for 0 < r < tq. 


3 Some properties of the entries of the pay¬ 
off matrix 

We start by considering the pay-off F’(^) of strategy ATFT against itself, 
with function F being given by (13). 

Proposition 1 (i) F{0) = R. 

(a) lima;^oo R{x) = P. 

(Hi) F is a decreasing function in [0,-|-oo). 

(iv) There exist positive constants Ki,K 2 such that 


r 


P K2 
- < — 


^2 


(17) 


(v) There exist positive constants such that 


K. 


< 


R-F{ 


< 


K. 


£i 


(18) 


Proof The hrst two items are direct consequences of (13). The third item 
follows easily by calculating the derivative of F and using both inequalities 
in ( pT| . 

Using polar coordinates (16) we get 

P(7^)-P 


£2 


1 (R - R) Sind + (S + T - 2R) COS0 

r (cos 0-|-sin 0)2 


By using (10) and 0 the function of 0 in the right-hand side is clearly 
strictly positive and continuous in the compact [0, |]. Letting Ki be its 
minimum and K 2 hs maximum, assertion (iv) is proved. 

Item (v) can be proved in an analogous way. ■ 
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Items (i), (ii) and (iii) in Proposition]^ prove that the pay-off of an ATFT 
against another ATFT may be any nnmber in (P, R) regardless of the small¬ 
ness of e. 

If g = 0, strategy G becomes TFT. As a conseqnence of this, see m, 


ai3(0)=a3i(0) = P(^). 

^2 


( 19 ) 


Also at g = 1 both formnlas for ai3 and 031 simplify and we obtain 

013(1) = P-|-(T — P)ei and 031(1) = R—{R — S)€i. (20) 

Other important properties of these same entries are: 

Proposition 2 (i) 013(g) > ctsiil) Vg G (0,1]. 

(ii) 013(g) ~ 03i(<?) 'is an increasing function in [0,1]. 

(iii) Ife is small enough, then both 013(g) and 031(g) are increasing functions 
in [0,1]. 

(iv) o'^3(0) 00 and ofy(O) 00. 


(v) 0)3(1) 0 and 033(1) 0. 

(vi) If e is small enough, and 033(g) are both negative in [0,1]. 

Proof After some easy manipnlations with ( [I^ and (15), we obtain 

ei(T- 5 )g 


ai3(g) - a3i(<?) = 


? + (l-?)(ei + e2) ’ 


which proves assertion (i). Differentiating the above eqnation proves (ii). 
To prove (iii), we dehne hrst an anxiliary variable 


X = q + { 1 - g)(ei 62) 


( 21 ) 


which leads ns to 


, , , 2 R-S-T- e3(T - P) - e2(P - S) 

®3iw) = D-- 




( 22 ) 
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Notice then that a'^^{q)/ei is a continuous function of e, positive at e = 0. As 
a consequence, is positive if ei > 0 , provided that e is small enough. 

Using (ii) the analog result is obtained for ai^{q). 

To prove (iv), we substitute g = 0, thus a; = ei + 62 , in (22). Using polar 
coordinates and 


"31 


we get 


(0) - 1 (>^ + ^ - 2 ^) cos^ e + (2R 
r 


S-T) sine 


(cos^ + sin 6 *)^ 


+ 0 ( 1 ) 


As the function of 6 multiplying 1/r is positive, then the result for 031 ( 0 ) is 
proved. Using again (ii), we prove the same for + 13 ( 0 ). 

The proofs of (v) and (vi) follow similar ideas. ■ 

Although formulas (14) and (15) are complicated. Proposition]^ tells a lot 
about these functions. In particular, properties (iv) and (v) show that both 
ai 3 and 031 grow very fast for q close to 0 and then saturate before q = 1 . 

To close this section, a simple and important 


Corollary 1 If e is small enough and a G (F(^),i? + (T — R)ei), then 
equation ais^q) = a has a unique root q in interval (0,1). Analogously, if 
[3 G (F(^), R — {R — S)ei), then 031 (g) = (3 has a unique root in (0,1). 


4 Locating the equilibria 

We start this section by studying the red equilibrium P123, the simplest 
among the equilibria in which only two strategies coexist, because its lo¬ 
cation is independent of the variable g, as shown by the following result. 


Proposition 3 The red equilibrium P123 is independent of q and always bi¬ 
ological. 

Proof Equating htnesses /i and / 2 , given by ([^, and writing X 2 = 1 — Xi, 
which is equivalent to 0:3 = 0 , we obtain 


3;i(Pi23) — 


1 


1 - 


T-P J_]_ 

P-S P-S 


fC^)-p ’ 

e2 


(23) 


which is indeed independent of g. Using (iv) in Proposition]^ we see that the 
denominator in the above equation is dominated by a positive term of order 
1/r. Thus xi(Pi 23 ) > 0 and as small as we want if e is small enough. ■ 
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The next result will be important when showing that the black equilib¬ 
rium will become biological for some intervals in q, because it states that the 
intercepts of lines 77 - 12 , '^•13 and 7723 appear always in the same order on L 3 . 
Notice the appearance for the hrst time of a hypothesis stating that forgive¬ 
ness q of individuals adopting strategy 3 must not be too close to 0. This will 
happen in other parts of this section and has the clear meaning that strategy 
3 must be more forgiving than strategy 1 for some of the results to be true. 

Theorem 1 (Order on L 3 ) If q E [(ei -|- € 2 )^'^^,!], then 0 < Xi(Pi 23 ) < 
Xl{P233) < Xi{Pi33) < 1 - 


Proof For ease of comparison, we may rewrite all three quantities in the form 
XiiPijs) = where formulas for the Cij will be presented. We will show 

that 0 < Ci 3 < C 23 < C 12 , from which the claim will be a trivial consequence. 
An easy calculation leads to 


a3i{q)-P{%) 

q{P-S) 


and 


C23 — 


a3i(g)-P-e2(T-P) 


g(P - 

and C 12 may be obtained in (1^. As 031 (g) is increasing and 031 ( 0 ) = P(f^), 
then Ci 3 > 0. If the ratio ^ is fixed and e is taken small enough, we obtain, 
using (ii) and (hi) in Propositionthat P(f^) > P + e 2 {T — P), thus proving 
that C 13 < C 23 for small enough e and g > 0 . 

Using (20) and, again, the fact that a 3 i(g) is increasing, we may see that, 
ifg> (el + e 2 )l/^ 


C 23 < 


P - (P-5)ei - P-e 2 (T-P) 
(6i + e2)i/2(P-A) 


which increases as when r —)• 0. On the other hand, by (23) and (iv) in 

Proposition [T| we see that C 12 increases as We conclude that £23 < £12 
for small enough e. ■ 

We may now dehne two numbers related to when the green and blue 
equilibria become biological: 
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Definition 4 According to Corollary^ 


ai3(g) = R 


has a unique root in (0,1). Let qgreen be this root. 

Let also 

_ p 

Qblue = _ p ( 2 ^) 

be the unique root of equation 023 = R. 

With these definitions we prove an important general result: 

Theorem 2 Let q G (e 2 ,1]. Then: 

• The blue line intereepts Li if and only q < quue and intereepts L 2 if 
and only if q E [qbiue,^]- In particular, the blue equilibrium is biological 
if and only if q < quue- 

• The green line intereepts Li if and only q < qgreen and intercepts L 2 
if and only if q E [qgreen, I] ■ In particular, the green equilibrium is 
biological if and only if q > qgreen- 

Proof After easy calculations we get 


X2{P23l) 

1 

“ 1 1 giP-s) 

^ 'P R-P-q(T-P) 

(25) 

Xi{P232) = 

1 

(26) 

a,,Pq)-P-e2{T-P) ’ 
R-P-q{T-P) 

XliPl32) 

1 

1 1 a.3i{q)-F{ei/e2) ’ 

o.l3{q)-R 

(27) 

2:2(.Pi3i) 

1 

1 (9-E2)(P-5) ’ 

ai3{q)-R 

(28) 


all in the form 1/(1+X), which will be in (0,1) if and only if the corresponding 
X is positive. In all four cases the proof that the necessary X is positive if 
and only if the respective condition on q is satisfied is trivial. In the cases 
related to the green line, we must use item (hi) in Proposition ■ 
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Besides quue and qgreen we will define a number which will signal the 
passage of the red line throngh the origin. In order to do that, let /i(g) be 
defined by 

h(?) = ai 3 (g) - P - q{T - P). (29) 

In terms of this new function we may easily obtain 


X2{Pl2l) 


h(g) 

h(?) + ^2{P — S) 


(30) 


and 


Xl{Pl22) 


_M?)_ 

l‘{‘l)-[F{ei/i2)-P-e2{T-P)] ' 


(31) 


which show that the red line passes through the origin of the {xi,X 2 ) plane 
whenever /i has a zero. 

The following lemma proves existence and uniqueness of such a zero: 


Lemma 1 Function fi defined by (29) has a single critical point q and a 
single zero qred in (0,1) such that qred > q. Furthermore, 


q IS a maximum 


point, p. is positive in (0, g^ed) and negative in {qred, !]■ 


Proof As p'{q) = a[fiq) — {T — P), then (iv) and (v) in Proposition imply 
that /i'(0) > 0 and /i'(l) < 0 if e is small enough. Then p has at least a 
critical point g G (0,1). Item (vi) in the same proposition proves uniqueness 
for g and that it must be a maximum point. 

As /r(0) = F{^) — P > 0, then pifi) > 0. And as p{l) = R — T — {T — 
R)ei < 0, then p has a single zero in (0,1) and this zero is located in (g, 1). 
The assertion on the signs of p follows from the fact p'{qred) < 0. ■ 

It is now time to start displaying important results in which the sign of 
Gi defined in (1^ plays an important role. The first thing to notice is that 


formula (14) for ai 3 (g) is notably simplified when Gi = 0. Solving equation 
ai 3 (g) = i? is trivial and we get, for Gi = 0, 


qgreen 


2R-S-T 

R-S 


If we calculate the difference between this value and quue 
identity 


2R-S-T 

R-S 


Qblue 


GfiT-R) 
{R-S){T-P) ’ 


we discover the 
(32) 
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which shows that qgreen and quue coincide when Gi = 0 . As ai^^qgreen) = R, 
we discover that qt{qgreen) = {quue —qgreen) (T — P), from which we can dednce 
that qred also coincides with qgreen and quue when Gi = 0 . 

If Gi 7 ^ 0, althongh more complicated, eqnation ais^q) = R leads only 
to a second-degree eqnation in q and a closed formnla for qgreen can also be 
obtained. If we solve the eqnation in the anxiliary variable x defined in (21) 
and notice that q and x differ by 0 {r), we prove in general that 


Qgreen 


2R-S-T 


Gi{T-R) 


R-S {R- S){2R- S -T) 


ei + 0 {r^), 


(33) 


with the interesting conseqnence that the exact calcnlated value of qgreen for 
Gi = 0 holds as a good approximation for qgreen even when Gi ^ 0 . 

By using the ideas above we can easily prove 

Theorem 3 (i) If Gi = 0 , then qgreen = Qbiue = Qred- 

('^^) then Qg reen ^ qbiue ^ qred’ 

(Hi) If Gi ^ 0 , then qgreen ^ qbiue ^ qred’ 


Equation (33) will be useful later to guarantee that qgreen does not tend 
to 0 when r —)■ 0. We will also need to prove the same for qred’ This is an 
easy consequence of the next result. 


Proposition 4 


qred qUue T 0{t) 


( 34 ) 


Proof Using the definition of /i ( 29 ) and ( 14 ), we may rewrite /i(g) = 0 as 
i?(l — £ 2 ) — ^1*5' ei[2i? — S — T -|- (T — i?)(ei -|- € 2 ) -h Gi^i] 1 Gief 


£1 - £2 


ei[2i? — S — T -|- (T — R)(ei -h € 2 ) + GiSi] 1 ^ 

1 - ei - €2 X 1 


£1 - £2 


= P 


X - ei - 62 
1-61-62 


(T-P), 


which solution in x will yield qred’ If we substitute 61 = 62 = 0 above, the 
solution is simply Xq = quue’ 

Substituting xq we may rewrite the above equation as g{x, e) = 0, with 


9 {x,e) = 


61 [2i? — S' — T -|- (T — R){£i + 62 ) G 161 ] 1 

X 

Gie? 


x^ 


1-61-62 

-|- Xq — X -j- 61 (T — S') -|- 62 ( 7 ^ — R) . 


18 














As 5 f(xo, 0 ) = 0 with |^(a:o,0) = — 1 + 0{r) ^ 0 for small enough r, the 
implicit function theorem proves that in some neighborhood of e = 0 the 
root X of g{x, e) = 0 is a differentiable function of e. Differentiability implies 
that this root is a; = Xq + 0{r). Noticing that q and x differ by 0(r) leads 


to (34). 


We will start justifying tables to with the case Gi > 0. Before that, 
a couple of technical results still independent of Gi. 


Lemma 2 Let go ^ ( 0 , 1 ] be fixed and independent of e. Then: 
• 031(90) ~ not tend to 0 when r —)■ 0 . 


Ifq^ [ 90 , 1 ], ^a;i(Pi32) = 0 {r). 


Proof Using variable x, defined in (21), and (15) we may write 

{ \ d I {R — T)ei 

031 ( 9 ) - F{ — ) = R- F{—) + -- 

€2 62 i — ei — 62 


ei 2R-S-T -[{T - P)ei + (P - 5 ) 62 ] 


1-61-62 


X 


+ 


Gie\ 


(1 - 61 - 62)x2 


Remember that if go is fixed, x does not tend to 0 when r —)■ 0. This means 
that, apart of the term R — F{^), the remaining terms in the right-hand side 
do tend to 0 when r —)■ 0. On the other hand, (v) in Proposition proves 
that R — P(f^) is positive and does not tend to 0. This proves the hrst part. 

In order to prove the second part, notice that 


A 

dq 


Xl{Pl32) 


0)3(9) 1 

Ti(9)-f’(S)) 

+ (P- oi 3 (g))a 3 i(g) 


013(9) -P + 031(g) -P(f;) 

2 


If g > qo , 013 (g) — P is 0(r). Moreover, by the hrst part of this Lemma, 
031(g) ~ does not tend to 0 when r —)■ 0. Thus the denominator in 

the expression above does not tend to 0 when r —0. In the numerator 
both terms are 0{r), as can be seen in ( 22 ) and its analog for 013 . As a 
consequence, the derivative of Xi(Pi 32 ) is 0 (r). ■ 


Proposition 5 //g G [g^ed, 1 ], then 


d 

dq 


{Xi{Pu2) - Xi{Pi32)) > 0 . 
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Proof Formulas for a:i(Pi 22 ) and 0 : 1 (^ 132 ) have already been given, see (31) 
and (27). 


Equation ( |M[ ) guarantees that qred does not tend to 0 as r —)■ 0. Then, 
by Lemma 1^ using e.g. go = ^/‘^Quue < Qred-, we conclude that 4 ^ 1 (^ 132 ) is 
0 {r) in [qred, !]• 

By an easy calculation, we have 


d [(T - P) - - P - i-AT - P)] 

122 ) ( J ) _ p _ _ P ) _ ^(,)|2 

It follows that 


d 

dq 


Xl{Pl22) > 


(T - P)- AAgred) 

ng) -P- t2(r- P) • 


(35) 


because in [qred, 1] we have /r(g) < 0, a'i 3 (?) < a'lsi^red) and F(|) - P - 
62 ( 7 " — P) > 0. In this last expression a[^{qred) is 0(r) and the denominator 
does not tend to 0. So there exists a positive constant C independent of r 
such that ^xi{Pi 22 ) > C. 

We conclude that for small enough e, ■^{xi{Pi 22 ) — Xi{Pi 32 )) > 0 for 

q G [qred, 1] • ® 

We can now prove our Erst result on the black equilibrium: 


Proposition 6 If Gi > 0, the black equilibrium is biological for all q G 
{qred, !]■ If Gi > 0 the conclusion extends also to q = qred- 

Proof First of all, if Gi > 0, by Theorem]^ we know that qred < Qgreen- For 
q = qred we then know that the green line intercepts the sides L 3 (Theorem 
and L 2 (Theorem]^ and then we must have a:i(Pi 32 ) < 0 for g = qred- But 
xi{Pi 22 ) = 0 at qred- By Propositionwe discover that a:i(Pi 22 ) > 2 :i(Pi 32 ) 
for g G {qred, !]• Comparing this order with the order on side L 3 given by 
Theorem we see that the red and green lines must cross at the interior of 
B for all g G {qred, !]• If Gi = 0 we already knew that the red and green 
lines crossed exactly at the origin for g = qred- But for Gi > 0, a;i(Pi32) < 0 
already at g = qred, and the lines cross in the interior. ■ 

Our task is now to End out when the black equilibrium Erst becomes 
biological and if it ever loses its biological status. We start with a general 
result: 
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Proposition 7 For any value ofGi and q = (ei + 62 )^'^^ we have X 2 {Pi 2 i) > 
X2{,Pld,l) ■ 


Proof The coordinates for the intercepts of the nij lines with xi 
all be written as 


X2{,Pijl) 


1 

l + bij{q) ■ 


where, after easy calculations, we get 

bi 2 {q) = and 613 (g) 

m) 


(g- 62 )(P-g) 
R - ai3(?) 


0 may 


(36) 


As ai3((ei + €2)^''^) — R - 0 (V^^) and fi((ei + €2)^''^) = R - P - 0 (VP), 
then by the above expressions our assertion is true. ■ 

The order just proved between 0 : 2 (^ 121 ) and a; 2 (Pi 3 i) at g = (ei + 62 )^^^ 
is reversed at g = g^ed if Gi > 0 . In fact, if Gi > 0 and g = g^ed, we have 
a^ 2 (Pi 3 i) > 0 , because g^reen > Qred, and a; 2 (Pi 2 i) = 0 . It turns out that the 
green and red lines must cross on Li for at least one g G ((ei + € 2 )^^^, gred)- 
We will show that this crossing is indeed unique, dehning quack- If Gi = 0 , 
we saw that the green and red lines crossed at g = qred- We will also show 
that no other crossing will happen if g G ((ci + 62)^'^^, qred)- 

If 0 : 2 (^ 121 ) — 3;2(Pi3i) were monotonic in ((ei + e 2 )^R, qred) fhe assertions 
in the preceding paragraph would be trivial. As this does not happen, we 
must work a bit harder, starting with 


Proposition 8 Equation 


p{<]) = (fi + £2)“''^ 


(37) 


has a single root q G [0,gred]; which is asymptotically given by 

(ei + 62)^/^ 


Q — Qblue 


T - P 


+ 0 {r) 


(38) 


In particular, q does not tend to d as r —)• 0. 


Proof Let g be the critical point of /i as in Lemma 0 As /i( 0 ) > (ei + 62 )^^ 
if e is small enough, and pi is increasing in [0,g], then equation (37) has no 
solution in that interval. On the other hand, as p, is decreasing in (g, g^ed] 


with p{qred) = 0, then (37) must have one root exactly in {q,qred)- 
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In order to obtain (38) we rewrite (37) using definition (29) along with 
(14) written in terms of variable x defined by (21). Putting ei = €2 = 0 we 

quue, which suggests us to define a new 
Substituting x = quue + 
in (37) and making several simplifications, we get that (37) is equivalent to 
H{r, y) = 0 , where 


obtain the approximate solution x 
auxiliary variable y as y = {x — qwu 


H{r, y) = cos^ 0(1 - r^/'^y - quue) 

cos 0[2i? - ^ - T - r cos 0(T - P) - r sin e{R - ^)] 

quue + 

+ r^/^[(T — S) COS0 + {T — R) sin0)] — (cos0 + sin0)^'^^[l — r(cos0 + sin0)] 
- {T-P)y. 


Repeating the argument with the implicit function theorem as in the proof 
of Proposition we obtain (38). ■ 

We now prove a monotonicity argument for X 2 (P’121) — (P’131) , but re¬ 

stricted to (g, qred)- 


Proposition 9 If Gi > 0 , ^ 2 (^ 121 ) — a; 2 (P’i 3 i) is a decreasing function of q 
in {q,qred)- 


Proof 

(p ) = (g - ^ 2 ){P - S)a[ 3 {q) + {R- ai3{q)){P - S) 
dq ^ [R-ai 3 {q) + {q - e 2 ){P - S)]^ 

shows that ^a: 2 (P’i 3 i) is negative and 0 {r) in {q,qred)- In order to conclude 
that, we are using the fact proved in Proposition that q does not tend to 
0 when r —0, which implies that both R — ai^i^q) and a'i^{q) are 0(r) for 
q > q. The denominator is of course 0(1) due to the (g — 62 )(P — S) term. 
For X 2 (Pi 2 i) we have 


^ 62 (P - S)p'{q) ^ p'{q) 

d,r'- W«) + e 2 (P-S)P e,_{P-S) 


2 


1 + 


€2(P-S) J 


In (g, qred), t^'i.q) = o-isiy) ~ {T — P) < —I{T — P), where we are using again 
that a[^{q) is 0(r). Also, 

1 1 sin 9 


1 + 


u{g) 

£2 (P-5) 


> 


> 


1 + 


M(g) 

€2{P-S) 


sin 0 


r-V2[sin0+ 


O I r~^/^{cos6-\-sm6)^/‘^ 

Sin u “r P—S 

> Pf sin 0 , 
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where K = max^g[0,7^72] (^sinO + > g. 

Finally, we obtain, for q G {q,qred), 

from which it turns out that ■^{x 2 {Pi 2 i) — X 2 {Pi 3 i)) < 0. ■ 

Putting together all known facts, we can now prove 

Theorem 4 (Black equilibrium, Gi > 0) IfGi > 0 andq G ((ei+e2)^^^, 1], 
there is a single value quack £ {q, Qred) such that the red, green and blue lines 
cross on the border of B. Moreover, the crossing is on Li. 

Proof By the properties of qt proved in Lemma [T| and noticing that 

2^2(^121) = , 

^ Viq) 

it is clear that the minimum of 0:2(^121) in [0,g] is attained at one of the 
boundaries of the interval. But as /i(0) = 0(1), and /i(g) = (ei + €2)^'^^, the 
minimum is attained at q and its value is thus 1 — 0(r^O)_ 

An easy calculation shows that the derivative of X2{Pi3i) is negative in 
[(ei + e2Y^‘^,qred)- If can be seen also that X2{Pi3i) = 1 — 0(1) at g = 
(ei + 62)^^^. Thus the maximum of X 2 {Pi 3 i) is less than the minimum of 
a;2(P’i2i) in [(ei + 62)^^^, g]. This proves that the red and green lines do not 
cross on Li for g G [(ci + 62)^^^, g]. 

On the other hand, they do cross for g somewhere in (g, q^ed) because we 
have already seen that at g = qred < Qgreen we have X2(Pi3i) > 0 = X2(Pi2i)- 
We have also just proved that the reverse holds at g = g. Uniqueness of this 
crossing in (g, qred) follows from Proposition]^ Uniqueness in ((ei + €2)^'^^, 1] 
is a consequence of Proposition ■ 

Having settled the question of the black equilibrium for Gi > 0, we 
remember that the question of whether the other equilibria are biological or 
not is already solved in Proposition and Theorem The results of which 
equilibria are biological for Gi > 0, all justihed, are summarized in Table 
The elements for justifying the equilibria results of Table for the case 
Gi = 0 were already proved. It remains for us just the task organizing them. 
First of all, in the case Gi = 0 we dehne quack to be equal to the common 
value quue = qgreen = qred- We then have 
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Theorem 5 (Equilibria for Gi = 0 ) If Gi = 0 , besides equilibria at the 
vertices of B and the red equilibrium, which are always biological, these are 
the biological equilibria at each interval: 

• The blue equilibrium is biological if and only q G [0,qbiack)- 

• If q > (ei + £ 2 )^^^; the green and black equilibria are biological if and 
only if qe {quack, !]■ 

Proof The assertions for the blue and green equilibria were already proved in 
TheoremIn Theorem]^ we have already proved that the black equilibrium 
is biological for q e {quackA]- The only thing remaining to be proved is 
that the red, blue and green lines do not cross on the border of B for q G 

((^1 + Qblack)- 

In fact, by Proposition!^ and the same argument in the proof of Theorem 
1^ we show that there is no crossing for q G ((ci + 62 )^^^, g). No crossing 
for q G (g, g^ed) is a consequence of Proposition Finally, for (g^ed, 1] the 
argument is Proposition So the red, green and blue lines only cross at the 
origin for g = quack- ■ 

The arguments necessary for proving validity of the equilibria results of 
Table are similar and simpler than the ones used for the other two cases, 
so that we will leave them to the reader. 


5 The dynamics 

Replicator dynamics was studied from the point of view of the theory of Dy¬ 
namical Systems in [16]. Zeeman addressed mainly the robust cases, where 
robust means cases in which the dynamics remains unchanged for arbitrarily 
small changes of parameters. In particular, Zeeman showed that it was pos¬ 
sible to classify the phase portraits of replicator dynamics by only knowing 
about the existence or not of equilibria at each face of the biological sim¬ 
plex, existence or not of an interior equilibrium, and the stability of each 
equilibrium. In the case of three strategies, the number of possible phase 
portraits was small enough so that each possibility could be shown. Zee¬ 
man’s work was continued in [2], which included also the non-robust cases, 
thus obtaining a complete classihcation of all possible phase portraits for the 
replicator dynamics with three strategies. As we will also be interested in 
some non-robust cases, in this section we will refer to the classihcation by 
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Bomze. In particular, we will see that, in most of the possible intervals and 
values of Gi in Tables [T] to our knowledge up to now can associate a single 
diagram in |2] with compatible dynamics. In these cases the dynamics is 
then fully determined. In some other cases, indicated in the tables, there 
were three compatible diagrams. The enumeration of the diagrams in our 
tables is the same as in [2]. The missing information, which led to doubt 
in determining the dynamics, is whether the black equilibrium is attractive, 
repulsive or neutral. Of course, for hxed values of parameters, it is an easy 
numerical task to linearize the dynamics around the black equilibrium, and 
by calculating eigenvalues of a 2 x 2 matrix, discover this information. But 
we do not have a general argument to show e.g. that the stability of the 
black equilibrium is the same for any choice of parameters in the last row of 
each of our tables. 

Another information necessary for reading our Tables [T] to is that the 
letter R in front of the number of a diagram in [2] means that one should 
take the corresponding diagram with all arrows reversed. In fact, the reader 
should notice that replacing matrix A by —A in equations Q has only the 
effect of reversing the orientation of all orbits. 

In order to be able to reproduce the results about the diagrams in our 
tables, we must know about the stability of the equilibria on each side of 
B whenever they are biological. On each side of B one of the strategies is 
absent and we only need to study the one-dimensional replicator dynamics 
for two strategies. Results for this case are rather trivial, see e.g. [HI p. 50], 
and only depend on the strategies being or not Nash equilibria. The results 
enumerated below are simple consequences of pairwise comparisons between 
elements of the pay-off matrix (§. 

1. In the absence of strategy 3, strategies 1 and 2 are both strict Nash 
equilibria. Thus, for the replicator dynamics restricted to side L 3 , the 
red equilibrium is unstable. Moreover, for any g G ((ci -|- 62 )^^^, 1] we 
can divide this side in four regions in which the htness ranking is the 
following: 

• Between E 2 and P 123 , we have /2 > /i > /s- 

• Between P 123 and ^ 233 - we have fi > f 2 > fs- 

• Between P 233 and P133. we have /i > /3 > / 2 - 

• Between P 133 and Pi. we have /3 > /i > / 2 - 
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In particular, in a neighborhood of P123, /a is the smallest fitness. This 
implies that /a < 0 in a neighborhood of ^123, with the consequence 
that orbits in the interior of B close to P123 must have X3 < 0 , and 
flow towards L3, which makes P123 a saddle point. This property is 
particularly important, because in some cases it is what allows us to 
discard some diagrams in P] otherwise compatible. 

2 . In the absence of strategy 1 , we have two possibilities: 

• If g < Qhiue) the blue equilibrium is biological, and as strategies 2 
and 3 are both strict Nash equilibria, then the blue equilibrium is 
unstable if the dynamics is restricted to the Li side. 

• If g > Qbiue, the blue equilibrium is not biological, and only strat¬ 
egy 2 is a Nash equilibrium. Then all orbits on Li must flow into 
P2. 

3 . In the absence of strategy 2 , we also have two possibilities: 

• If ( 61 - 1 - 62 )^'^^ < g < qgreen, then the green equilibrium is not biolog¬ 
ical, and between strategies 1 and 3 , only 3 is a Nash equilibrium. 
All orbits on L2 must flow into P3. 

• If g > qgreen, then the green equilibrium is biological. Because nei¬ 
ther strategy 1 , nor strategy 3 are Nash equilibria, then the green 
equilibrium is asymptotically stable when dynamics is restricted 
to L2. 

An example can help clarify how we have obtained the results on the ta¬ 
bles regarding the compatible diagrams and type of evolution of cooperation. 
We take as an example the hrst line in all three tables. We know that for 
g > (ei -|- 62)^^^, but not too large, regardless of Gi we will have as biological 
equilibria only the three vertices, and the red and the blue equilibria. By 
the above reasoning, Ei must be a saddle point, whereas P2 and P3 are at¬ 
tractors, the red equilibrium is a saddle with outgoing orbits on L3, and the 
blue equilibrium has outgoing orbits on Li. Point Q is not in the biological 
region. 

Among the diagrams in | 2 ] not a single one is compatible with the above 
situation. But if we reverse the orbits, then diagrams 37 and 38 seem com¬ 
patible. The only one which remains compatible when we take into account 
that interior orbits close to the red equilibrium must flow towards L3 is 38 R. 
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Figure 1: Phase portrait and graph of the fractions Xi^ X 2 and x^ as functions 
of time for the following parameter values: T = 5,-R = 4, P = 2,S' = 0, 
ei = .05, 62 = .10, q = .40. The initial condition for the graphs is {xi,X 2 ) = 
(.18, .76). Observe the full evolution of cooperation. 


In Figure [T] we show a plot of some numerically calculated orbits for a choice 
of parameters in one of the cases leading to diagram 38R. Notice that all or¬ 
bits below the separatrix joining the blue and red equilibria lead to survival 
only of strategy G. This justihes why we have full evolution of cooperation. 
For orbits below the separatrix, but very close to the red equilibrium P 123 
we can see the occurrence of the phenomenon in the experiment of [ 12 ]: an 
initial population with majority of ALLD, some ATFT and quite a few G 
individuals evolves to a population where only the G individuals are present, 
after passing through a transient in which the ATFT are almost the entire 
population. The phenomenon is illustrated by the graphs of fractions xi, X 2 
and X 3 as functions of time in Figure 

As already mentioned, in the cases at the last line of each Table [T] to 
we could not find a rigorous argument for proving which of diagrams 12, 12R 
and 13 is the correct one. 

In the case Gi > 0 (Table |^, we already know for interval {qbiuei Qgreen) 
that the only compatible diagram is 15R, in which the black equilibrium is 
unstable and, consequently, there is no evolution of cooperation. It is not 
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reasonable that increasing q will foster cooperation. In fact, larger values of 
q will make the G individuals more susceptible to exploitation by ALLDs. 
Thus the natural conjecture is that if Gi > 0 and q G {qgreen, 1] the black 
equilibrium will still be unstable and no evolution of cooperation will happen. 
If this conjecture is true, then the associated diagram must be 12R. The 
conjecture is supported also by numerical calculation of the eigenvalues of 
the linearized dynamics around the black equilibrium. 

In the Gi < 0 case we know that there is full evolution of cooperation until 
g = Qgreen and ouly partial evolution for qgreen < q < quack, due to the green 
equilibrium destabilizing E^. For larger values of q the black equilibrium 
appears and we have no knowledge on its stability. Numerical calculation 
of eigenvalues suggests that in the case Gi < 0 the black equilibrium is 
asymptotically stable and will attract all orbits in a region of positive area, 
which means weak evolution of cooperation and that the correct diagram 
should be 12 . 

Finally, in the Gi = 0 case we may expect a situation intermediate be¬ 
tween the other two cases. Numerical calculations suggest that the real part 
of the eigenvalues of the linearized dynamics around the black equilibrium 
may be 0. Numerically calculated orbits around the black equilibrium seem 
to be closed. The correct diagram would be 13, in which the interior equi¬ 
librium is a center, and evolution of cooperation would also be weak. 

6 Conclusions 

We have proved that the results of the computer experiment in [T2] are true 
for a simplihed version of that experiment in which, instead of 100 reactive 
strategies, we only have the three more prominent ones in the experiment: 
ATFT, ALLD and G. More precisely, if we dehne 


QGTFT — Qblue} 


( 39 ) 


then for q G ((ei -f 62 )^'^^, gcTFr) there exists a region C C B with positive 
area such that the orbit of the replicator dynamics for any initial condition 
in C will converge to E^, i.e. only the G individuals will survive. 

Equation (39) should be thought as the rigorous version of the (|^. In 
fact, we say that ([^ is not precise because [ 6 ], neglecting 0 (r) terms, dehned 
GTFT as strategy r(l, g) with q close to the value in ([^. Our equation ( [M| 
shows in a precise sense that (§ is indeed a good approximation. We also 



proved that is exact if and only if Gi = 0, and that which between qgreen 
and quue is the minimnm depends on the sign of Gi. 

We have also partially nnderstood the popnlation dynamics for valnes of 
q larger than qcTFT- We have seen that in some cases some weaker forms of 
cooperation evolntion will still hold, bnt we have also seen that if Gi > 0 and 
q £ {quue^ qgreen) HO evolntion of cooperation is possible, becanse for almost 
all initial conditions only ALLD individnals will snrvive. The same sitnation 
probably holds also for q > qgreen and Gi > 0, bnt probably it does not hold 
for q > quack and Gi < 0. 
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